On Macroscopic Complexity and Perceptual Coding

The theoretical limits of ’lossy’ data compression algorithms are considered. The complexity of an object as seen by a macroscopic observer is the size of the perceptual code which discards all information that can be lost without altering the perception of the specified observer. The complexity of this macroscopically observed state is the simplest description of any microstate comprising that macrostate. Inference and pattern recognition based on macrostate rather than microstate complexities will take advantage of the complexity of the macroscopic observer to ignore irrelevant noise. The quantification of information Information theory in its modern form originated from Claude Shannon’s[22] usage of Gibbs’ entropy formula to describe communication channels: S = −k ∑ Pi logPi (1) This formula originally applied to an ensemble of microscopic states[8], and the analytic form of expected log-probability describes the entropy of systems and their representations whether classical or quantum in nature. In the context of quantum mechanics, it becomes the von Neumann entropy of the state density matrix, S = −trace(p log p). The story goes that it was actually von Neumann who suggested the term ’entropy’ to Shannon for his information function, for two reasons: ’In the first place your uncertainty function has been used in statistical mechanics under that name, so it already has a name. In the second place, and more important, nobody knows what entropy really is, so in a debate you will always have the advantage.’ Entropy has the units of the logarithm of action[13] Shannon showed that, in the absence of Boltzmann’s constant, k, entropy quantifies the number of bits of data needed to identify a sample from some distribution. It quantifies the amount of choice or uncertainty that must be overcome in order to invoke the axiom of choice and select a specific element from a set. The Shannon Entropy,